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Introduction 


What Is Student Agency? 


Student agency, or the ability to manage one’s learning, can have significant effects on academic 
achievement as students take an active role in seeking and internalizing new knowledge. Students 
who believe that knowledge can grow over time perform better on IQ tests than students who 
believe intelligence is invariable (Cury, Elliot, Da Fonseca, & Moller, 2006); students with a 
growth mindset are more likely to set academic goals focused on mastering content, rather than 
setting goals focused on achieving a particular test score or course grade (Cury et al., 2006); 
and students who set mastery-oriented goals tend to process information in a deeper and more 
organized fashion than those who set performance-oriented goals (Elliot, McGregor, & Gable, 
1999). In addition, the skills and behaviors associated with student agency are positively 
related to college and career outcomes because students are able to direct their own learning 
and transfer the knowledge they learned in the classroom to new settings (National Research 
Council, 2012). 


However, less is known about the instructional strategies teachers can use to develop agency in 
students. To learn more about the practices that support student agency, the American Institutes 
for Research (AIR) worked with four New Tech Network (NTN) high schools in three states (Exhibit 1) 
for the study Maximizing Student Agency: Implementing 
and Measuring Student-Centered Learning Practices to 
determine which teacher practices help or hinder the 
development of student agency, and whether these 


Exhibit 1. Participating New Tech 
Network Schools 


practices are effective across educational contexts 
and with different student subgroups. 


NTN schools use project-based learning to empower and 

challenge students to learn and succeed, collaborate 

and communicate, and engage in the world around them. 

A critical component of their approach is student agency, 5 6e) 


or students’ capabilities to manage their own learning 
and be successful in school. Teachers from each of 
the four NTN high schools participated in a networked 
improvement community (NIC) to test and refine strategies 


to support student agency. 
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Maximizing Student Agency: Implementing and Measuring 
Student-Centered Learning Practices 


AIR’s study team used a mixed-methods approach’ to address key research questions aligned 
to larger focus areas for the study (Table 1). 


Table 1. Focus Areas and Research Questions 


Focus Area Primary Research Question(s) 


What practices do teachers employ to provide feedback to 
Teacher practices designed to promote students on their performance that assist with the development 
student agency of student agency? 


How do teachers use data to inform their practices? 


Contextual factors influencing the promotion What contextual factors do teachers view as facilitators of or 
of student agency challenges to implementing these practices? 


How well do student survey questions measure student agency? 


Were the measurement properties of the agency scales consistent 
over time and across the student subgroups? 


Lessons learned about surveying student Are there significant subgroup differences in measures of 
agency over time student agency? 


How does student agency change during the school year? 


Do changes in student agency during the school year differ 
between subgroups of students? 


Networked Improvement 


A key feature of the study was the use of a networked improvement community (NIC)—a group of 
high school teachers from four high schools across three states who wanted to further promote 
student agency in their classrooms. From spring 2017 to spring 2018, the NIC members: 


Convened to define the root causes preventing them from promoting student agency in 
their classrooms, 


Agreed that they could make changes in their practice to create opportunities for students 
to demonstrate agency, 


Selected a change idea (i.e., practice) to promote agency that they would systematically test, 


Developed formative measures to track the extent to which students demonstrated 
agency, and 


Implemented the change idea during Plan-Do-Study-Act (PDSA) cycles (Exhibit 2) to 
determine the extent to which their change ideas led to increased student agency. 


1 Study methodology is detailed in the technical appendix. 
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A more detailed description of the NIC’s efforts, AIR’s supports, and related resources follows. 


Exhibit 2. PDSA Cycles 


Act Plan 
@ Based on data analysis, decided @ Use research or other local 

how to proceed. evidence to identify a change idea 
m Participants may want to adapt, — ~ that addresses one or more root 


causes of the problem. 


@ Develop an implementation plan 
for the change idea. 


adopt, abandon, or expand on 
the change idea. 


Study 

= Collect data to monitor the progress 
of the change idea. 

= Analyze data and interpret findings 
to learn about the successes and 
challenges of the change idea. 


Do 
= Implement the change idea. 


AIR facilitated five NIC convenings during the 2017-18 school year. Between these meetings, AIR 
met with teachers individually or as a school-based large group to support key decision points 
throughout the improvement process. Exhibit 3 outlines a timeline of events. 


Spring 2017 


The NIC launched and AIR introduced the NIC’s guiding principles: 
m™ Teachers are central to student empowerment, 


™ Guiding the development of student agency is complex and almost impossible to do in 
isolation, and 


m™ Teaching can be continuously improved. 
The NIC reviewed the definition of student agency driving the study. We shared two sources of 
data from focus groups conducted prior to the NIC’s launch: the Menu of Teacher practices and 


challenges for promoting student agency. The convening concluded with teachers from each 
school beginning to articulate their goals for student agency. 


Summer 2017 


AIR examined the evidence base available for the Menu of Teacher Practices generated by the 
focus group results. The purpose of this step was to inform the NIC of specific examples of 
practices, as appropriate, when the NIC participants selected their change ideas to test. 
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Fall 2017 
October 2017 


The NIC convened to discuss the benefits of a NIC and 
the importance of a shared, measurable goal for their 
work together (i.e., an aim statement), as well as to 
revisit the shared barriers or root causes that prevent 
teachers from promoting agency. The NIC then discussed 
which of these root causes teachers have the power to 
change and what practices might be potential drivers for 
that change. The NIC participants further refined their 
thinking about these topics through online discussions 
on the NIC’s online Groupsite platform before they 
were finalized. 


November-—December 2018 


NIC members selected the change ideas they wanted 
to test. They completed a “Change Idea Hypothesis 
Worksheet” where they articulated the practice, why they 
selected it, what the practice would look like/sound like, 
what the immediate student response and longer term 
student outcomes might be, and what measures would 
be used to determine if their hypotheses were correct. 
In the worksheet, the teachers also articulated their 
plans for when the change idea would be implemented 
(i.e., before, during, and/or after school), how often it 
would be implemented during a given week, duration 

of the practice (i.e., how many weeks before the 
aforementioned changes could be realized), and 

how the teacher planned to track whether s/he 
implemented the change idea as planned. 


The NIC participants decided to test seven out of the 17 
practices listed on the Menu of Teacher Practices. Three 


Exhibit 3. NIC Timeline 


Spring 2017 


March 
Teacher focus groups 


April 
Networked improvement 
community launch 


April/May 
Developed Menu of Teacher Practices 


Summer 

Identification of evidence base 
for strategies within the Menu 
of Teacher Practices 


Fall 
Student baseline survey 


October 

Analysis of root problems and 
development of aim statement 
December 


Identification of areas of influence 
needed to achieve the aim 


Winter 2018 


January/Februrary 

NIC discussion of change ideas 
and hypotheses 

Februrary/ March 

PDSA cycles 


May 
Debriefing of PDSA cycle data 


of the practices were “student opportunities”; three were “student-teacher collaborations”; 


and one was a “teacher-led approach.” 


AIR provided feedback on the Change Idea Hypothesis Worksheets, met with the individual teams 
and/or participants, and discussed how to further specify their hypotheses and measurement plans. 
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Winter 2018 
January 2018 


AIR and the NIC participants codeveloped measures aligned to the goals in the NIC members’ 
Change Idea Hypothesis Worksheets and encouraged members to share their change ideas on 
the NIC Groupsite. 


Spring 2018 
February—May 2018 


NIC participants implemented their change ideas and met with AIR to examine early PDSA cycle 
results, troubleshoot measurement challenges, discuss whether the change idea needed to be 
adapted, and reflect on their experiences in the improvement process. 


May-June 2018 


NIC participants began to send AIR their final PDSA cycle data. AIR requested that NIC participants 
also complete a Change Idea Planner. This form was designed to capture the change idea as it 
was implemented in the PDSA cycle. The NIC thought of this form as a “recipe” for others to 
consider following in their classroom once the teams have more data to decide if these practices 
increase student agency. The NIC held its final convening so that each team could hear from the 
others about their change ideas, early results, lessons learned, and next steps. AIR also shared 
early results from the student agency surveys (described in the following section). The teams 
enthusiastically shared their desire to keep this work going. 


Data Sources 


Data? for the current study came from three primary sources: survey data, focus group data, and 
PDSA cycle data collected by the NIC. 


Student survey data. In fall 2017 and spring 2018, the research team administered a student 
survey that included measures of student agency as well as student demographic information. In 
fall 2017, we collected survey data from 184 students attending the four participating schools.*® 
A second survey was administered to 385 students (including 132 of the students who took the 
fall survey) in spring 2018. Although the fall 2017 survey was limited to students in classes with 
NIC teachers, to facilitate analyses that look more closely at differences between subgroups of 
students, all students within participating schools were invited to participate in the spring 2018 


ND 


Information about survey samples, response rates, and data analysis methods is detailed in the technical appendix. 
This number excludes survey respondents who were removed from the data for a variety of reasons, including 
missing student name, students reporting that they did not take a class with a NIC teacher (in the fall), 


duplicate records where students reported twice about the same class, and missing data for all 54 items 
associated with the student agency scales. 


w 
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survey. Overall, we analyzed data from 437 unique survey respondents. 


A list of the student agency measures included in the student survey, along with references to the 
sources of the measures, is provided in Table 2. Each construct was measured with between four 
and nine survey items, and responses to survey items ranged from 1 (disagree) to 4 (strongly 
agree). For each survey construct, we calculated a scale score by averaging responses to relevant 
survey items. Averages and standard deviations for the student agency measures also are 
provided in Table 2.4 


Table 2. Student Agency Constructs, Sources, and Example Items 


Fall Spring 


Standard 
Deviation 


Standard 


Construct Source Example Item Deviation 


Average 


Average 


In general, | think that | can 


hen, Gully, ; 
Self-efficacy Shen Gulls achieve goals that are 3.07 0.60 3.03 0.61 
Eden, 2001 . 
important to me. 
New ideas and projects 
poeante Puc uonh sometimes distract me from 2.69 0.68 2.56 0.74 
of interest? Quinn, 2009 : 
previous ones. 
Perseverance Duckworth & 
| finish wh | begin. 2: : 2.84 67 
of etor Quinn, 2009 inish whatever | begin 88 0.66 8 0.6 
Eeclisiof Levenson, 1981 | Can Pretty much determine = 547 g57 2.89.55 
control what will happen in my life. 
. An important reason why | do 
M Midgl l., . : 
ae Deep abe my classwork is because | like 2.67 0.72 2.60 0.75 
orientation 2000 . 
to learn new things. 
| ask myself questions to 
nee Pintrich & make sure | understand the 
iti 2. .67 2. 64 
COBTIING ; DeGroot, 1990 —_— material | have been studying ee On ug ne 
self-regulation ane 
in this class. 
Consortium on 
If-regulated | ide ti d 
peal ated Chicago School seed ie : Bes 2.79 0.72 267 0.70 
g Research, 2009 t 
Future Consortium on What | learn in class is 
: . Chicago School necessary for success in the 3.07 0.80 2.89 0.78 
orientation 


Research, 2009 


future. 


*Items in the perseverance of interest construct were reverse-coded so that higher values indicate a higher level of perseverance. 


4 As described in the technical appendix, we found that the measurement properties of several agency measures 
improved after removing one or two survey items. The averages and standard deviations in Table 2 were 
calculated after removing these survey items. 
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In addition to the measures of student agency, student surveys included questions that capture 
key student background information, including gender, race/ethnicity (White, Black, Hispanic, or 
“other” racial/ethnic group), socioeconomic status, and English learner (EL) status (i.e., whether 
a language other than English is spoken at home). 


Teacher survey data. The research team administered surveys to 58 teachers in fall 2017 and 
65 teachers in spring 2018. In the teacher survey, teachers were asked about the frequency 
with which they engaged in practices associated with increasing student agency® with most of their 
students. In addition, teachers were asked about how many students in their school (none, some, 
about half, most, or nearly all) have different types of learning opportunities. Finally, the teacher 
survey included survey items that allowed us to measure key aspects of the school setting (e.g., 
teachers’ commitment to the school, perceived program coherence, instructional improvement 
culture, self-efficacy for teaching). 


Teacher focus group data. In spring 2017, the research team conducted focus groups with 

AO teachers to discuss their definitions of student agency, goals for agency in their classroom, 
practices and opportunities designed to promote agency, data currently collected on student 
agency, and facilitators of and barriers to agency. In spring 2018, the research team conducted 
a second round of teacher focus groups to gather additional data about perceptions of student 
agency as well as NIC activities. 


Student focus group data. In spring 2018, AIR conducted student focus groups at each of the 
study schools, with a total of 48 students. Students were asked to provide feedback on the following: 
= Definitions of student agency, 


® Opportunities they have been provided to employ agency, 


Instructional practices their teachers have used this year 
(aligned to the Menu of Teacher Practices), 


Skills those practices have helped develop, and 


Ideas for improvement. 


PDSA cycle data. The research team aided 25° teachers participating in the NIC in choosing 
specific change ideas to implement within the classroom to increase student agency. The team 
guided NIC teachers in completing PDSA cycles to develop change ideas, implement them, test 
related outcomes, and refine them. Examples of the types of data collected by the teachers 
include student responses to brief surveys, students’ grades, workshop attendance, and work 
resubmission rates. 


5 The practices included in the teacher survey reflected the instructional practices identified during the spring 
2017 focus groups and outlined within the Menu of Teacher Practices. 


® During the course of the study, there were changes to the composition of the NIC (e.g., long-term substitute 
taking over a NIC classroom for a maternity leave, teachers leaving the NIC). 
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Study Findings 


In this section, we summarize the study findings by the study’s three focus areas defined earlier: 
teacher practices designed to promote student agency; contextual factors influencing student 
agency; and lessons learned about surveying student agency over time. 


Teacher Practices Designed to Promote Student Agency 


The first overall focus area for the study was identifying practices that teachers used to promote 
agency. Data from the teacher focus groups, teacher surveys, PDSA cycles, and NIC meetings were 
used to answer two primary research questions: 


= What practices do teachers employ to provide feedback to students on their performance 
that assist with the development of student agency? 


= How do teachers use data to inform their practices? 


What practices do teachers employ to provide feedback to students on their performance that 
assist with the development of student agency? 


Key Findings 


™ Created from discussions during teacher focus groups, the Menu of Teacher Practices identified 17 instructional 
practices that teachers use to develop agency at the four study schools. These practices fall into three general 
categories: student opportunities, student-teacher collaboration, and teacher-led approaches. 


= NIC participants tested seven of the 17 practices included in the Menu of Teacher Practices. Three of the 
practices fell in the category of student opportunities, three practices represented student-teacher collaborations, 
and one was a teacher-lead approach. 


@ In both fall 2017 and spring 2018, the most common practice was to develop personal relationships with 
students to better understand their agency strengths, needs, and motivators. 


™ The percentage of teachers reporting that they model agency skills to demonstrate those skills to students in a 
meaningful context or provide students with tools, strategies, and resources to coach them toward mastery of 
agency skills doubled between fall 2017 (14% for each practice) and spring 2018 (30% and 28%, respectively). 


During the spring 2017 teacher focus groups, participants were asked to identify practices they 
use in their classroom to help develop agency. Responses were coded and analyzed to develop 
the Menu of Teacher Practices (see Exhibit 4; also Appendix A). The menu includes brief 
descriptions of 17 teacher practices that fall within three general categories: 


= Student opportunities. Practices in which students are the primary decisionmakers in how 
the instructional practice is implemented. 


= Student-teacher collaboration. Practices in which students and teachers work together to 
implement the instructional practice. 


= Teacher-led approaches. Practices in which teachers are the primary decisionmakers in 
how the instructional practice is implemented. 
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Exhibit 4. Menu of Teacher Practices on Student Agency 


ALY ae 
#40 Student Opportunities 
Choice. Teachers provide students with opportunities to make choices about the content and process of their work. 


Group Work. Teachers provide students with opportunities to work in groups to learn and practice agency necessary 
for group success. 


Harnessing Outside Opportunities. Teachers provide students with opportunities to demonstrate agency outside the 
classroom and make connections between outside agency and its application in the classroom. 


Revision. Teachers provide students with opportunities to revise assignments or tests after they have received feedback. 


Student Self-Reflection. Teachers provide students with opportunities to self-reflect using journals, logs, or other 
structured templates or tools. 


Student-Led Instruction. Teachers provide students with opportunities to demonstrate agency by leading instruction 
on a particular skill or concept. 


ma Student-Teacher Collaboration 


Developing Relationships. Teachers develop personal relationships with students to better understand their agency 
strengths, needs, and motivators. 


Feedback. Teachers provide students with feedback and scaffold the process of students asking for feedback. 
Goal Setting. Teachers help students set goals to complete coursework while improving agency. 


Individual Conferences. Teachers hold one-on-one meetings with students to discuss elements of student agency 
and its relationship to academic work. 


Student Voice. Teachers provide students with opportunities to contribute to and provide feedback on key decisions 
in the classroom. 


4 Teacher-Led Approaches 


Assessment. Teachers design formative and summative assessments to evaluate student agency and/or to provide 
students with extrinsic motivation to build agency. 


Direct Instruction. Teachers provide explicit instruction to develop skills related to student agency. 
Modeling. Teachers model agency to demonstrate it to students in a meaningful context. 
Positive Reinforcement. Teachers provide positive reinforcement for demonstration of agency. 


Scaffolding. Teachers provide students with tools, strategies, and resources to help scaffold students toward mastery 
of agency. 


Verbal Cues. Teachers provide brief spoken prompts in real time to highlight or remind students of behaviors that 
demonstrate agency. 


To learn more about how these practices are used in the participating schools, AIR administered 
teacher surveys. In the fall and spring surveys, teachers were asked about the frequency with 
which they used specific practices to promote student agency. Table 3 presents the percentage of 
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teachers who reported using these practices in the fall and spring with most of their students 
more than three times per week. In both fall and spring, the most common practice was to develop 
personal relationships with students to better understand their agency strengths, needs, and 
motivators. Some of the least frequently reported practices in both fall and spring included holding 
one-on-one meetings with students to discuss elements of student agency and its relationship to 
academic work, providing students with extrinsic motivation to build agency skills, and providing 
explicit instruction to develop skills related to student agency. For the majority of practices listed 
in Table 3, we did not observe a large change in the percentage of teachers reporting that they 
use practices more than three times per week with most of their students between the fall and 
spring survey administrations. However, the percentage of teachers reporting that they model 
agency skills to demonstrate those skills to students in a meaningful context or provide students 
with tools, strategies, and resources to coach them toward mastery of agency skills doubled 
between fall 2017 (14% for each practice) and spring 2018 (30% and 28%, respectively). 


Table 3. Percentage of Teachers Who Reported Using Practices More Than Three Times per Week 
With Most of Their Students, Overall and for Teachers Who Did and Did Not Participate in the NIC 


Fall 2017 Spring 2018 


All NIC Non-NIC All NCTE TLC 


Teachers Teachers Teachers [eachers ‘teachers Teachers 


Practice (n=58) (n=33) (4°) (n=65) (ihe) ‘(ToL 0)) 


Make connections between outside agency 


. rae 20.7 15.2 28.0 17.2 20.0 13.8 
and its application in the classroom. 


Revise assignments or tests after they 


have reechedded back. 13.8 15.2 12.0 2419 22.9 20.7 


Self-reflect using journals, logs or other 


20.7 15.2 28. 14.1 17.1 10. 
structured templates or tools. 0 5 8.0 0.3 


Lead instruction on a particular skill 
or concept. 


15.5 12.1 20.0 22.2 32.4 10.3 


Contribute to and provide feedback 


ee 19.0 21.2 16.0 23.4 25.7 20.7 
on key decisions in the classroom. 


Develop personal relationships with 
students to better understand their 56.9 57.6 56.0 57.8 54.3 62.1 
agency strengths, needs, and motivators. 


Guide students in the process of asking 


for feedback. 27.6 24.2 32.0 26.6 20.0 34.5 


Help students set goals to complete 
coursework while improving their agency 20.7 15.2 28.0 18.8 14.3 24.1 
to do so on their own. 


Hold one-on-one meetings with students 
to discuss elements of student agency 10.3 9.1 12.0 9.4 8.6 10.3 
and its relationship to academic work. 
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Fall 2017 Spring 2018 


IN NIC -Non-Nic _ All NIC Non-NIC 
Teachers Teachers Teachers Teachers Teachers Teachers 
Practice (i teks)) (n=33) (is) ‘(i Tsk o})) (i he)) (ri 0)) 
Design formative and summative 12.1 15.9 8.0 10.9 143 6.9 
assessments to evaluate student agency. 
pune students ie extrinsic motivation 105 9.4 12.0 12.5 143 103 
to build agency skills. 
Provide explicit instruction to develop skills 10.5 12.5 8.0 141 171 103 
related to student agency. 
Model agency skills to demonstrate those 14.0 18.8 8.0 29.7 374 20.7 


skills to students in a meaningful context. 


Provide positive reinforcement for 


f . 29.8 34.4 24.0 26.6 31.4 20.7 
demonstration of agency skills. 


Provide students with tools, strategies, 
and resources to coach them toward 14.0 18.8 8.0 28.1 34.3 20.7 
mastery of agency skills. 


Provide brief spoken prompts in real 
time to highlight or remind students of 28.1 37.5 16.0 30.2 38.2 20.7 
behaviors that demonstrate agency. 


Although the study’s research questions did not specify a comparison of the practices employed 
by teachers who participated in the NIC and teachers who did not participate in the NIC (non-NIC 
teachers), the study team explored whether specific practices were more common among NIC 
teachers, and whether they observed changes over time in practices implemented by NIC and 
non-NIC teachers. Because all teachers who responded to the surveys taught in schools that 
promoted student agency, several practices were employed by both types of teachers (e.g., revising 
assignments or tests after they have received feedback; developing personal relationships with 
students to better understand their agency strengths, needs, and motivators; providing students 
with extrinsic motivation to build agency skills). However, several practices were employed 
more frequently by teachers participating in the NIC, such as providing positive reinforcement for 
demonstration of agency skills; modeling agency skills to demonstrate those skills to students ina 
meaningful context; providing students with tools, strategies, and resources to coach them toward 
mastery of agency skills; and providing brief spoken prompts in real time to highlight or remind 
students of behaviors that demonstrate agency. 


In contrast, we also observed differences in which non-NIC teachers appeared to use specific 
practices more frequently than NIC teachers: guiding students in the process of asking for feedback, 
helping students set goals to complete coursework while improving their agency to do so on their 
own, and holding one-on-one meetings with students to discuss elements of student agency and 
its relationship to academic work. Finally, while we did not observe substantial declines (of more 
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than 5 percentage points) in the percentage of NIC teachers reporting the frequent use of specific 
practices between the fall and spring, we did observe decreases of this magnitude for the 
practices of making connections between outside agency and its application in the classroom 
(14 percentage points), self-reflection using journals, logs or other structured templates or tools 
(18 percentage points), and leading instruction on a particular skill or concept (10 percentage 
points) among non-NIC teachers. 


We also explored the extent to which teachers’ reported practices were employed at the school 
level. Exhibit 5 presents the percentage of teachers who reported that specific opportunities were 
available for less than half, about half, or most/nearly all of the students in their schools in spring 
2018.’ Results suggest that some opportunities were more prevalent for students in participating 
schools (e.g., 84% of teachers in the spring reported that most or nearly all of their students work 
in groups to learn and practice agency skills necessary for group success) while other opportunities 
were less prevalent (40% of teachers reported that most or nearly all of their students make 
connections between outside agency and its application in the classroom). 


Exhibit 5. Percentage of Teachers Reporting the Schoolwide Use of Strategies to Promote Student 


Agency in Spring 2018 
Demonstrate agency by leading instruction 3 
: f 11.1 55.6 
on a particular skill or concept. 
Self-reflect using journals, logs, or other 
63.5 
structured templates or tools. 
Revise assignments or test after they have 
. ra R:) 
received feedback. 
Make connections between outside agency 
rer yy) 39.7 
and its application in the classroom. 
Demonstrate agency outside the classroom 194 41.9 
(in the school or in the community). : 
Work in groups to learn and practice agency 84.4 
skills necessary for group Success. : 
Make choices about the content and 172 563 
process of their work. 
0% 20% 40% 60% 80% 100% 


@ Less than half Bi About half i Most or nearly all 


’ The survey was also administered in fall 2017. Results were similar between the beginning and end of the 
school year. 
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How do teachers use data to inform their practices? 


Key Findings 


™ NIC teachers developed their own measures, including surveys, attendance trackers, and grade trackers, to test 
outcomes associated with their change ideas related to student agency. 


Within the NIC, teachers developed their own measures, including surveys, attendance trackers, 
and grade trackers, to test the effectiveness of change ideas related to student agency (Table 4). 
To keep the schools anonymous, we refer to them as School A-D. 


Table 4. PDSA Cycle Measures 


School Change Idea Measure 


Administer a student self-reflection at the end of each week : 
School A Weekly self-reflection survey 
to measure mastery and resources used. 


Individual Assessment of Knowledge 
Provide students with additional resources and feedback so and Thinking (IAKT) survey 


hool B 
sone they can revise and improve their work. 
Growth mindset survey 
Writing agency survey 
Waive zero-grade policy for students who seek out extra help. 
School C Workshop attendance 
Provide students with personalized verbal feedback. Feedback survey 
Resources survey A 
Resources survey B 
Provide students more choice in support resources. 
School D Resources survey C 


Agency survey 


Provide one-on-one conferencing on IAKT assignments. Student writing grades 


Although we illustrate a few examples of how teachers used student data below, detailed tables 
of all the PDSA cycle data are provided in Appendix B. Several teachers wanted to learn what 
resources students use most often or find most helpful. For example, School B developed and 
administered a survey for their students to complete approximately once per month. Teachers 
were able to use these data to see that students reported conducting independent research 
online and responding to teacher feedback contributed a lot to their learning, while teacher-led 
workshops and conferences did not contribute a lot to their learning (See Table 5). 
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Table 5. School B IAKT Survey Responses 


February (n=93) ETC (1 ba? 074) May (n=178) 


Number Percent Number Percent Number Percent 


To what extent did responding to teacher feedback left in Echo contribute to your learning while completing 


this IAKT? 
Did not use 4 4.3 27 13.4 24 13.5 
Did not contribute 4 4.3 6 3.0 6 3.4 
Contributed some 30 32.3 68 33.7 60 33.7 
Contributed a lot 55 59.1 101 50.0 88 49.4 


To what extent did responding to peer feedback contribute to your learning while completing this IAKT? 


Did not use 32 34.4 53 26.2 47 26.4 
Did not contribute 11 11.8 19 9.4 18 10.1 
Contributed some 34 36.6 41 20.2 35 19.7 
Contributed a lot 16 17.2 89 44.1 78 43.8 


To what extent did using the resources available in the project briefcase contribute to your learning while 
completing this [AKT? 


Did not use 14 14.9 20 9.9 18 10.1 
Did not contribute 4 43 6 3.0 6 3.4 
Contributed some 35 37.2 99 49.0 88 49.4 
Contributed a lot Al 43.6 17 38.1 66 37.1 


To what extent did conducting independent research online contribute to your learning while completing this IAKT? 


Did not use 7 td 13 6.4 12 6.7 
Did not contribute 3 3.2 0 0.0 0 0.0 
Contributed some 16 17.0 20 9.9 18 10.1 
Contributed a lot 68 72.3 169 83.7 148 83.2 


To what extent did teacher-led workshops contribute to your learning while completing this IAKT? 


Did not use 37 41.6 106 52.5 95 53.4 
Did not contribute 15 16.9 AT 23.3 Al 23.0 
Contributed some 27 30.3 35 17.3 30 16.9 
Contributed a lot 10 11.2 14 6.9 12 6.7 


To what extent did teacher-led conferences contribute to your learning while completing this [AKT? 


Did not use 42 46.7 107 53.0 96 53.9 
Did not contribute 14 15.6 41 20.3 36 20.2 
Contributed some 25 27.8 40 19.8 34 19.1 
Contributed a lot 9 10.0 14 6.9 12 6.7 
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Teachers then used this information to examine student responses within their classes, study 
student response patterns across classes, and determine whether to continue to test the change 
idea. Teachers saw that students generally reported an increase in their perceived connections 
between taking ownership of their learning (i.e., using resources, conducting independent research, 
and participating in workshops) and their performance on the IAKT. This increase occurred one 
month after implementing the change idea. The increase flattened out and declined approximately 
one month later, perhaps because of the time of year (i.e., students had been finalizing their 
projects and were not learning as much new content). Teachers were eager to test this change 
idea starting in fall 2018 to see if students may be able to perceive the connection between their 
agency and their performance earlier in the school year and sustain it over the course of the 
school year. 


Based on the results, the teachers also plan to be more explicit in how they talk about their 
supports for their students. For example, the teachers hypothesized that students may not 
consider their informal, frequent conferencing as teacher-led conferences. The teachers plan to 
be more intentional about the language they use in the classroom so that students are able to 
more accurately define and identify their learning experiences. 


Several other teachers provided more feedback to students, with the goal of helping students 
achieve mastery of a content area or skill. One teacher in particular introduced the change idea of 
having one-on-one conferences during which the student took an active role in recording teacher 
feedback as well as their own insights. To test the effectiveness of this increased feedback, the 
teacher tracked student grades on writing assignments over the course of the semester (See 
Table 6). 


Table 6. School D Student Writing Grades 


Score of 1 Score of 2 SY e0) (ot 0) io) 


Number Percent Number Percent Number Percent 


Development 60 19.7 15 20.3 0 0.0 
aan PSSteAMeM Geant 35 47.3 36 48.7 3 41 
Language 52 68.9 23 31.1 0 0.0 
Assignment 1 Draft Development 56 74.3 14 18.9 5 6.8 
(n=75) Organization 55 73.0 17 23.0 3 4.1 
ake Language 53 70.3 19 25.7 3 44 
Development 43 56.8 26 35.1 6 8.1 
re mnie Organization 43 56.8 17 23.0 15 20.3 
Language 43 56.8 21 28.4 11 14.9 
Assignment 2 Draft Development 60 19.7 13 17.6 2 2.7 
(n=75) Organization 52 68.9 20 27.0 3 4.1 
Apt 26 Language 55 73.0 19 25.7 1 14 
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Score of 1 Score of 2 Score of 3 


Number Percent Number Percent Number Percent 


Development 60 19.7 12 16.2 3 4.1 
Assignment 2 Final i 
(n=75) Organization 52 68.9 14 18.9 9 12:2 
Language 52 68.9 19 25.7 4 5.4 
Assignment 3 Draft Development 50 66.2 24 32.4 1 1.4 
(n=75) Organization 45 59.5 26 35.1 4 5.4 
Moy 2 Language 44 58.1 28 37.8 3 4.1 
Development 20 27.0 25 33.8 29 39.2 
eel bine) Organization 25 33.8 23 31.4 26 35.1 
Language 25 33.8 24 32.4 25 33.8 


Note. Writing assignments were scored on a scale from 1 to 3, with 3 being the highest or best score. 


These data helped the teacher to compare students’ baseline writing assignments (when students 
received written feedback from their teacher) to those assignments completed while the change 
idea was being implemented. Overall, the percentage of students scoring 3s on aspects of their 
writing increased over the semester. In addition, the percentage of students showing growth, or 
increased scores, between their draft and final versions of assignments increased over the semester. 
These data suggested to the teacher that taking time to provide individual feedback is valuable 
because it is correlated with improved student writing skills. 


Contextual Factors Influencing the Promotion of Student Agency 


The second overall focus area for the study was identifying factors that influence the use of 
instructional practices that promote agency. Data from the teacher focus groups were used to 
answer the following research question: 


= What contextual factors do teachers view as facilitators of or challenges to implementing 
these practices? 


What contextual factors do teachers view as facilitators of or challenges to implementing 
these practices? 


Key Findings 


™ Teachers identified supportive schoolwide processes, teacher collaboration, and professional development 
as the primary facilitators of implementation of instructional practices that promote agency. 


= Teachers identified limited time, student awareness and exposure to certain instructional practices, and 
educator perception and capacity as the primary challenges they see in implementation of instructional 
practices that promote agency. 
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Facilitators 


Focus group participants identified three primary facilitators in deepening the use of instructional 
practices that promote student agency: supportive school processes, teacher collaboration, and 
professional development. 


Schoolwide processes. The primary facilitator identified by teachers were the schoolwide 
processes in place at NTN schools. NTN schools follow a comprehensive school model in which 
student agency is up front and center. NTN has agency rubrics that identify along a continuum the 
ability to develop and reflect on growth mindset and demonstrate ownership over one’s learning.® 
In addition, each school had existing collaborative structures that support the collaborative 
development of projects. 


Teacher collaboration. Across each of the four schools, focus group respondents highlighted teacher 
collaboration as a key facilitator. Teacher collaboration provided opportunities for increased 
communication about effective (and ineffective) instructional strategies and collaborative problem 
solving that allows teachers to share strategies across grade levels and content areas. 


Professional development. Ongoing professional development to build the capacity of teachers 
also was identified as a key factor in supporting the use of instructional practices that develop 
agency. The role of the teacher in NTN schools is often that of a facilitator, which is a shift from 
the role of the teacher in a traditional classroom. Teachers reported professional development 
opportunities as ways to build new, effective facilitation skills. 


Challenges or Barriers 


Across the four schools, teachers noted a variety of challenges or barriers. The most commonly 
mentioned challenges were a lack of time to focus on agency development, limited student 
awareness or demonstration of agency, and educator perceptions and capacity. 


Time. Teachers identified a general lack of time to focus on promoting agency as one challenge. 
The pressure to focus on content standards that are often aligned to educator evaluation systems 
resulted in some teachers feeling that the ultimate instructional priority was on ensuring proficiency 
on state assessments. Time also came into play as a challenge when teachers discussed the 
amount of time needed to prepare resources and manage formative data. 


Student awareness. Teachers across the four schools shared a perception that students are not 
always prepared to take advantage of opportunities to demonstrate agency. Because NTN schools 
begin with Grade 9, teachers perceived that when students reach their building, they have had 
limited exposure to instructional practices such as choice or student-led instruction. Therefore, 
they are underprepared to engage fully in these practices when implemented in the classroom in 
NTN schools. 


Educator perceptions and capacity. Even within the participating schools’ structure and culture 
that have a specific focus on agency, teachers reported that some of their colleagues have 
perceptions that limit their use of agency-supporting instructional practice. In addition, some 
identified capacity gaps in terms of technological and pedagogical skills that would support 
improved implementation of some of the practices. 


8 https://newtechnetwork.org/resources/new-tech-network-agency-rubrics/ 
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Lessons Learned About Surveying Student Agency Over Time 


The third overall focus area for the study was learning more about surveying student agency 
over time and examining how well the measures performed with different subgroups of students. 
Data from the teacher focus groups, student surveys, and NIC meetings were used to answer 
five primary research questions: 


How well do student survey questions measure student agency? 


Were the measurement properties of the agency scales consistent over time and across 
student subgroups? 


Are there significant subgroup differences in measures of student agency? 
How does student agency change during the school year? 


Do changes in student agency during the school year differ between subgroups of students? 


How well do student survey questions measure student agency? 


Key Findings 


For measures of future orientation, locus of control, and metacognitive self-regulation, measurement of the 
scales improved after removing one or two items. The rest of the measures of student agency had effective 
measurement properties within the study sample. 


The measure of perseverance of interest did not relate strongly with other components of student agency, which 
is likely due in part to the way in which survey questions were worded. 


To examine change in survey measures over time and compare levels of student agency across 
student subgroups, it was necessary to examine the measurement properties of the student 
agency scales. First, we performed confirmatory factor analyses (CFAs) to ensure that the 
survey questions accurately measured intended constructs of student agency (e.g., self-efficacy) 
among the students in our study. Results of these analyses indicated that, for measures of future 
orientation, locus of control, and metacognitive self-regulation, measurement of the scales 
improved after removing one or two items. The rest of the measures of student agency had 
effective measurement properties within the study sample. See the technical appendix for more 
details. 


After calculating scale scores by averaging survey responses across relevant survey items for 
each of the measures of student agency, we examined interrelationships between the different 
measures of student agency. The correlation matrix for these measures is provided in Table 7. 
The eight measures are all intended to measure underlying components of student agency, so it is 
expected that the measures will be positively correlated with each other. As shown in Table 7, 
the measure of perseverance of interest did not relate strongly with other components of student 
agency. It is likely that this was at least partly due to the way in which survey questions were 
worded. Although survey questions associated with other measures of student agency were 
positively worded, with positive responses indicating a higher level of student agency, questions 
related to perseverance of interest were negatively worded. The research team reverse-coded the 
perseverance of interest measure, so that higher values indicate a higher level of student agency, 
but it appears that the wording of the survey questions led to systematic differences in patterns 
of student responses. 
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Table 7. Correlations Between Measures of Student Agency 


Measure SE PI PE LC Ke) MSR SRL FO 
Self-efficacy (SE) 1 
Perseverance of interest (Pl) 0.01 1 
Perseverance of effort (PE) 0.69 -0.04 1 
Locus of control (LC) 0.60 -0.13 0.62 1 
Mastery orientation (MO) 0.58 -0.05 0.59 0.54 1 
Metacognitive self-regulation (MSR) 0.64 -0.07 0.61 0.61 0.70 1 
Self-regulated learning (SRL) 0.68 0.08 0.62 0.58 0.66 0.73 1 
Future orientation (FO) 0.51 -0.02 0.45 0.45 0.60 0.56 0.62 1 


Were the measurement properties of the agency scales consistent over time and across the 
student subgroups? 


Key Findings 


Between five and eight of the student agency measures worked equally well across subgroups of students, with 
the largest number of issues observed by gender and socioeconomic status (SES) level. 


The analysis suggests there are only a few instances where student agency constructs did not work equally well 
across student subgroups. 


In addition to examining the overall measurement properties of the student agency constructs, 
it was necessary to determine whether the measurement properties were consistent over time 
and across various subgroups of students (e.g., across subject areas, demographic traits, grade 
levels). Results of these analyses confirmed that the ways in which survey responses related to 
one another within student agency measures did not differ across survey administrations, allowing 
us to examine change in levels of student agency from fall to spring.° 


Across the student agency measures, we did not observe a consistent pattern of differing 
measurement properties across multiple subgroups of students. However, we found a few 
instances where the measurement properties of student agency measures were not equal 
across different student subgroups (please see the Technical Appendix for detailed findings): 


Subject area. Measures of locus of control and self-regulated learning did not work equally 
well across academic subjects. Specifically, one survey item was not as strongly related to 
the locus of control construct for English language arts (ELA) and social studies classes as 
it was for other classes, and another survey item was not as strongly related to the locus of 
control construct in mathematics or social studies as it was for other classes. In addition, 
two survey items were not as strongly related to the self-regulated learning construct in 

ELA or interdisciplinary classes as they were for other classes. Overall, however, the 
majority of survey items worked equally well across subject areas for both constructs. 


° The research team did find evidence that survey items related to future orientation may have been interpreted 
differently by students in the fall and the spring. Please see the technical appendix for detailed findings. 
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Grade level. The measure of locus of control did not work equally well across grade levels. 
In particular, two survey items were not as strongly related to the locus of control construct 
for Grade 9 students as they were for students in higher grade levels, while two survey 
items were not as strongly related to the locus of control construct for Grade 10 students 
as they were for Grade 9 students. Because the majority of the locus of control survey 
items did not work equally well across grade levels, we would not recommend comparing 
levels of locus of control by grade level. 


Race/ethnicity. Measures of perseverance of interest and mastery orientation did not 
work equally well for White and non-White students. In particular, most of the survey 
items were not as strongly related to the perseverance of interest construct for White 
students as they were for non-White students, suggesting that we should not compare 
levels of perseverance of interest across racial/ethnic groups. In addition, results suggest 
that some of the survey items associated with mastery orientation were not interpreted 
similarly by White and non-White students, making comparisons across racial/ethnic 
groups problematic. 


Gender. Measures of perseverance of interest, locus of control, and future orientation 
did not work equally well for male and female students. Results suggest that some of the 
survey items associated with these constructs were not similarly interpreted by male and 
female students, making comparisons by gender problematic. 


SES. Measures of perseverance of effort, locus of control, and metacognitive self-regulation 
did not work equally well for higher SES students (who reported having 100 or more books 
in their homes) and lower SES students. Results suggest that some of the survey items 
associated with these constructs were not similarly interpreted by higher SES students 
and lower SES students, making comparisons by SES problematic. 


Overall, these findings show that between five and eight of the student agency measures worked 
equally well across subgroups of students, with the largest number of issues observed by 
gender and SES (with three problematic student agency measures within each of these subgroup 
comparisons). Therefore, it is only in the minority of instances where student agency constructs 
did not work equally well across student subgroups that we should use caution when interpreting 
subgroup differences in levels of and changes in student agency over time. For all other instances 
that were not found to be problematic, comparisons can be made across student subgroups. 


Are there significant subgroup differences in measures of student agency? 


Key Findings 


Findings from the fall and spring reveal somewhat different patterns of subgroup differences, likely due to the 
smaller number of survey respondents in the fall. 


The fall survey results found subgroup differences by SES and subject area, while the spring survey results 
found subgroup differences by EL status and SES. 
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The study team examined whether levels of student agency measures in the fall and spring 
significantly differed across student subgroups defined by grade level, subject area, gender, race/ 
ethnicity, EL status, and SES. Findings from the fall and spring reveal somewhat different patterns 
of subgroup differences. While the figures below illustrate significant subgroup gaps in student 
agency, detailed tables of all regression results are provided in the technical appendix. 


Fall survey data. Among the 175 students who responded to the fall survey and provided 
demographic information, we found subgroup differences by SES and subject area. Students 
from higher SES families (defined as having at least 100 books in their homes) reported 
significantly higher levels of mastery orientation, metacognitive self-regulation, self-regulated 
learning skills, and future orientation relative to lower SES students (See Exhibit 6).*° In addition, 
we observed subject area differences in reported levels of perseverance of interest and self- 
regulated learning (with higher reports in science relative to social studies); locus of control, 
mastery orientation, metacognitive self-regulation, future orientation (with higher reports in ELA 
relative to social studies); and perseverance of effort (with higher reports in ELA and science 
relative to social studies; see Exhibit 7).** 


Exhibit 6. Differences in Student Agency in Fall 2017 by Socioeconomic Status (SES) 


Future orientation 3.4* 

3.0 

Self-regulated learning 3.1* 
Metacognitive self-regulation 3.0* 
Mastery orientation 3.0* 
Locus of control 3.0 

3.0 
Perseverance of effort aa 3.0 
Perseverance of interest 
Self-efficacy 3.1 
1 2 3 4 


@ Higher SES students @ Lower SES students 


* Indicates that difference between higher SES and lower SES students is statistically significant at the .05 confidence level. 


1° Differences in metacognitive self-regulation by SES should be interpreted with caution because results of statistical 
tests suggest that survey items may have been interpreted differently by higher SES and lower SES 
students (see the technical appendix for more details). 


11 Differences in locus of control and self-regulated learning by subject area should be interpreted with caution 
because two survey items from each scale did not work equally well across subject areas (See the technical 
appendix for more details). 
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Exhibit 7. Differences in Student Agency in Fall 2017 by Subject Area 


3:1* 


Future orientation 


Self-regulated learning 


Metacognitive self-regulation 


Mastery orientation 


Locus of control 


Perserverance of effort 


Perseverance of interest 


Self-efficacy 


1 2 3 4 
@ Interdisciplinary MScience WeEnglish mm Math B Social Studies 


* Indicates that value is significantly higher than the value for social studies, the reference category, at the .05 confidence level. 


Spring survey data. Among the 354 students who responded to the spring survey and provided 
demographic data, we found subgroup differences by EL status and SES. First, EL students 
reported lower levels of self-efficacy, locus of control, mastery orientation, metacognitive self- 
regulation, self-regulated learning, and future orientation than non-EL students (see Exhibit 8). 
In addition, students with higher levels of SES reported higher levels of self-efficacy, perseverance 
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of effort, mastery orientation, self-regulated learning, and future orientation relative to lower 
SES peers (see Exhibit 9).12 


Exhibit 8. Differences in Student Agency in Spring 2018 by EL Status 
Future orientation 
Self-regulated learning 
Metacognitive self-regulation 
Mastery orientation 

Locus of control 

Perseverance of effort 


Perseverance of interest 


Self-efficacy 


1 2 3 4 
@ Non-EL students @ EL students 


* Indicates that difference between EL students and non-EL students is statistically significant at the .05 confidence level. 
Exhibit 9. Differences in Student Agency in Spring 2018 by SES 

Future orientation 

Self-regulated learning 

Metacognitive self-regulation 

Mastery orientation 

Locus of control 

Perseverance of effort 

Perseverance of interest 


Self-efficacy 


1 2 3 4 
W@ Higher SES students @ Lower SES students 


* Indicates that difference between higher SES and lower SES students is statistically significant at the .05 confidence level. 


12 Differences in perseverance of effort by SES should be interpreted with caution because results of statistical 
tests suggest that survey items may have been interpreted differently by higher SES and lower SES 
students (see the technical appendix for more details). 
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These findings show a consistent SES advantage in reports of student agency across the fall 
and spring survey administrations. In contrast, we did not observe consistent gaps in student 
agency by race/ethnicity, gender, or grade level in either the fall or the spring. It is possible that 
discrepancies between survey administrations (e.g., consistent differences by EL status observed 
in the spring survey data but not the fall survey data) are due to the smaller number of survey 
respondents in the fall. 


How does student agency change during the school year? 


Key Findings 


™ Student-level measures of student agency did not significantly change during the school year, with two 
exceptions: levels of perseverance of interest significantly declined from the fall to the spring, while levels of 
metacognitive self-regulation significantly increased over time. 


To examine changes in measures of student agency during the school year, we administered 
surveys to 132 students in both fall 2017 and spring 2018. We calculated changes in student 
agency measures by subtracting students’ fall values of student agency measures from their 
spring values. Our results revealed that student-level measures of student agency did not 
significantly change during the school year with two exceptions: levels of perseverance of 
interest significantly declined from the fall to the spring, while levels of metacognitive self- 
regulation significantly increased over time (see Table 8). 


Table 8. Changes in Measures of Student Agency From Fall 2017 to Spring 2018 


Construct Change Over Time Standard Error p-value 
Self-efficacy 0.072 0.045 0.107 
Perseverance of interest -0.216 0.105 0.039 
Perseverance of effort 0.051 0.087 0.555 
Locus of control -0.022 0.053 0.669 
Mastery orientation 0.091 0.108 0.397 
Metacognitive self-regulation 0.133 0.059 0.025 
Self-regulated learning 0.035 0.066 0.589 
Future orientation -0.051 0.055 0.361 


NIC teachers shared anecdotal evidence of changes they perceived in student agency. During the 
spring 2018 focus groups, teachers were asked to reflect on the year and share examples of 
emerging or developing student agency*’ over the course of the year. Teachers from all four 
schools commented that students appeared to improve at seeking support in ways that were 
aligned with the goals of agency development. Teachers noted that students asked for feedback 
with the intention to improve drafts and recognized that seeking support is “a practice of highly 


13 The participants tended to talk about specific skill development and demonstration of agency simultaneously. 
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successful students” instead of a sign of lower ability. Teachers observed students exhibiting 
self-reliance in seeking resources and supports independently, and asking each other for help 
before turning to the teacher for support. Teachers in three schools noted that students use the 
feedback built into the class design to genuinely learn from mistakes and to master the content 
instead of turning in work to consider it “done.” While not an overall theme across the focus groups, 
individual teachers provided examples of student skill development that included increases in 
student confidence, collaboration with peers, communication, and time management. 


Do changes in student agency during the school year differ between subgroups of students? 


Key Findings 


™ With the exception of grade level, the study team did not observe consistent patterns in subgroup differences 
in changes in measures of student agency during the school year. 


™ Grade 11 students experienced greater increases in persistence of effort and self-regulated learning than 
Grade 9 students, and Grade 10 students experienced significantly larger decreases in future orientation than 
Grade 9 students, during the school year. 


™ Because fall levels of student agency measures did not significantly differ by grade level, these findings suggest 
that rates of change in student agency measures may not be consistent across grade levels. 


In addition to estimating changes in student agency among all students in the sample, within the 
sample of 132 students who took both the fall and spring survey, we examined whether changes 
during the school year significantly differed for different subgroups of students. With the exception 
of grade level, we did not observe consistent patterns in subgroup differences in changes in 
measures of student agency during the school year. We found that Grade 11 students experienced 
greater increases in persistence of effort and self-regulated learning than Grade 9 students, and 
that Grade 10 students experienced significantly larger decreases in future orientation than 
Grade 9 students, during the school year (see Exhibit 10). Because fall levels of student agency 
measures did not significantly differ by grade level, these findings suggest that rates of change 

in student agency measures may not be consistent across grade levels. 
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Exhibit 10. Changes in Student Agency Measures Between Fall 2017 and Spring 2018, by Grade Level 


Future orientation 


0.4* 
Self-regulated learning 


0.4 
Metacognitive self-regulation 


Mastery orientation 


Locus of control 


0.5* 
Perseverance of effort 
Perseverance of interest 
Self-efficacy 
-1.5 -1.0 -0.5 0 0.5 1.0 1.5 


W@ Grade 11 W@ Grade 10 @ Grade 9 
* Indicates that value is significantly higher than the value for Grade 9 students, the reference category, at the .05 confidence level. 


In addition, we found a few instances, as follows, where changes over time significantly differed 
between student subgroups for one or two of the student agency measures. Detailed results 
for all analyses can be found in the technical appendix. 


Subject area. Students experienced smaller increases in future orientation during the 
school year in ELA and science classes than in social studies courses (the reference 
category), though this may be due in part to the fact that fall levels of future orientation 
were significantly higher for students reporting on these classes. Also, students experienced 
greater decreases in self-efficacy during the school year in mathematics than in social 
studies courses, despite the fact that we did not observe differences in fall levels of 
self-efficacy by subject area. 


Gender. Female students experienced smaller increases in perseverance of effort during 
the school year relative to male students, despite the fact that we did not observe gender 
differences in fall measures of perseverance of effort. 


EL status. Increases in metacognitive self-regulation were significantly smaller for EL students 
relative to non-EL students even though fall levels of metacognitive self-regulation did not 
significantly differ by EL status. 
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Conclusion 


Teacher Practices that Support Development of Student Agency 


The teachers who volunteered to participate in the NIC brought with them varying perspectives 
on what it means to promote student agency. These perspectives seemed to influence the 
expectations they set and opportunities they created for their students to demonstrate agency 
in their classrooms. The NIC created an opportunity for teachers and researchers to learn from 
this variation in teacher perspectives. 


Through engaging in a networked improvement community, teachers built their capacity to articulate 
their practices so that others might replicate them, predict which short- and long-term outcomes 
would improve as a result of the change idea, and map out data sources to measure those 
outcomes. This exercise was difficult for many, but through scaffolded conversations with the AIR 
team, the NIC participants learned how to be more purposeful in making connections between 
their instructional changes and desired student behaviors. 


The networked improvement approach allowed teachers to develop their own formative measures 

to track the results associated with those instructional changes. They learned how to look at data 
and decide whether the formative measures were gathering the types of information they were 
most interested in learning about. They were then able to share change ideas and challenges with 
one another through the NIC convenings. 


44 When it comes to implementing a change idea, | was really excited at the way 
that we were able to start as a group and come up with a broad range of ideas, 
work with other schools and see what their thoughts were and then decide as a 
team if we wanted to all do the same change idea.... | felt like everybody got on 
board...we had set times, we knew every two weeks we were going to get back 
together and review things. JJ 

— NIC Teacher 


Use of Student Agency Measures 


This study raises a number of implications as researchers and practitioners work to measure 
student agency and identify instructional practices that are effective in promoting student agency. 
First, we found that current survey measures of student agency did not always perform equally 
well across student subgroups (e.g., by subject area or SES), so we caution their use across 
educational contexts. Researchers and practitioners should take the time to investigate whether 
the measures they intend to use are interpreted similarly by different subgroups of students. 
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Second, with the exceptions of perseverance of interest and metacognitive self-regulation, we did 
not see significant changes in student agency over time using current survey measures. In fact, 
observed declines in perseverance of interest between the fall and spring may be due to the fact 
that the spring survey was administered at the end of the school year, when students were eager 
to start their summer vacations. In addition, in focus groups, NTN teachers mentioned “survey 
fatigue” when discussing the spring 2018 survey administration. Researchers and practitioners 
should consider this inability to capture short-term changes in student agency, the timing of data 
collection, and implications of survey fatigue when selecting measures for future studies. One 
possible solution is using an improvement science approach which would allow for multiple 
checks over time using multiple, practical measures within the classroom, which may result 

in the collection of more actionable data on student agency. 


Third, we found that the measure of perseverance of interest did not relate strongly with other 
components of student agency, likely due in part to the way in which survey questions were worded. 
Although survey questions associated with other measures of student agency were worded, with 
positive responses indicating a higher level of student agency, questions related to perseverance 
of interest were negatively worded. In the future, researchers and practitioners should consider 
rewording all questions within a student agency survey to have a common direction. 
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Appendix A. Menu of Teacher Practices 
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In April 2017, approximately 40 teachers from four New Tech Network (NTN) high schools participated in a series of focus groups 
designed to collect information on the practices they used to build student agency among their students. The data collected from 
the focus groups were utilized to develop the Menu of Teacher Practices on Student Agency. 


Three categories provide the organizational framework for the menu: 1. Student Opportunities, 2. Student-Teacher Collaboration, 
and 3.Teacher-Led Approaches. 


The menu includes brief descriptions of 17 teacher practices that fall within each of these three categories, and key elements that 
teachers identified as being part of each practice. 


Menu of Teacher Practices on Student Agency 


TA. oe 
hit Student Opportunities 
Choice. Teachers provide students with opportunities to make choices about the content and process of their work. 
Group Work. Teachers provide students with opportunities to work in groups to learn and practice agency necessary for group success. 


Harnessing Outside Opportunities. Teachers provide students with opportunities to demonstrate agency outside the classroom and make connections 
between outside agency and its application in the classroom. 


Revision. Teachers provide students with opportunities to revise assignments or tests after they have received feedback. 
Student Self-Reflection. Teachers provide students with opportunities to self-reflect using journals, logs, or other structured templates or tools. 


Student-Led Instruction. Teachers provide students with opportunities to demonstrate agency by leading instruction on a particular skill or concept. 


ia Student-Teacher Collaboration 

Developing Relationships. Teachers develop personal relationships with students to better understand their agency strengths, needs, and motivators. 
Feedback. Teachers provide students with feedback and scaffold the process of students asking for feedback. 

Goal Setting. Teachers help students set goals to complete coursework while improving agency. 

Individual Conferences. Teachers hold one-on-one meetings with students to discuss elements of student agency and its relationship to academic work. 


Student Voice. Teachers provide students with opportunities to contribute to and provide feedback on key decisions in the classroom. 


4 Teacher-Led Approaches 


Assessment. Teachers design formative and summative assessments to evaluate student agency and/or to provide students with extrinsic motivation 
to build agency. 


Direct Instruction. Teachers provide explicit instruction to develop skills related to student agency. 

Modeling. Teachers model agency to demonstrate it to students in a meaningful context. 

Positive Reinforcement. Teachers provide positive reinforcement for demonstration of agency. 

Scaffolding. Teachers provide students with tools, strategies, and resources to help scaffold students toward mastery of agency. 


Verbal Cues. Teachers provide brief spoken prompts in real time to highlight or remind students of behaviors that demonstrate agency. 


/18 
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The Menu of Teacher Practices on Student Agency was produced with support from Jobs for the Future’s Student-Centered Learning Research Collaborative 
and its funders. It was designed to be a living document that will continue to be revised as teachers pilot new practices and refine existing practices 
relevant to building student agency. 
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CHOICE: Teachers provide students with opportunities to 


make choices about the content and process of their work. 


Key Elements of Choice 


Choice of Content: Students have the opportunity to choose a topic 


or content area that builds personal relevance. 


Choice of Method: Students have the opportunity to choose an 
approach to meeting the requirements of an assignment. 


Choice of Skill: Students have the opportunity to choose their role in 
a group, the skills they are seeking to develop, and/or extra supports 
for skills that they find challenging. 


Choice of Engagement: Students have the opportunity to choose 
their level of participation and engagement in a task. 


GROUP WORK: Teachers provide students with opportunities 
to work in groups to learn and practice agency necessary 
for group success. 


Key Elements of Group Work 


Clear Roles and Responsibilities: Teachers facilitate group success 
by working with students to clearly define each group member's roles 


and responsibilities. 


Clear Expectations: Teachers work with group members to establish 
goals and benchmarks so that all group members have a common 


understanding of the task. 


Assessment: Teachers assess elements of group work, such as 
communication and collaboration, often incorporating a dimension 
of self-reflection and peer feedback. 


HARNESSING OUTSIDE OPPORTUNITIES: Teachers provide 
students with opportunities to demonstrate agency outside 
the classroom and make connections between outside 
agency and its application in the classroom. 


Key Elements of Harnessing Outside Opportunities 
Embedding Personal Relevance: Teachers provide students with 
opportunities to work on projects that are personally meaningful 
and foster conditions for students to demonstrate agency. 


Making Connections: Teachers illustrate connections between 
student actions and the extent to which they demonstrate agency. 


REVISION: Teachers provide students with opportunities to 
revise assignments or tests after they have received feedback. 


Key Elements of Revision 

Providing Opportunities to Revise Before Grading. Teachers embed 
the revision process into their project timeline, providing students with 
feedback and an opportunity to revise before grades are provided. 
Providing Opportunities to Revise After Grading. Teachers provide 
students with the option to revise and resubmit an assignment or test 
if they are not satisfied with the grade they receive. 

Providing Student-Led Opportunities to Revise. Teachers provide 
students with opportunities to collect feedback and make revisions 
if they choose to do so. 


STUDENT SELF-REFLECTION: Teachers provide students 
with opportunities to self-reflect using journals, logs, or 
other structured templates or tools. 


Key Elements of Student Self-Reflection 


Regular Self-Reflection on Coursework: Teachers provide opportunities 
for students to reflect on their coursework each day, identify challenges, 
and plan for next steps. 

Self-Reflection on Student Agency: Teachers provide opportunities for 
students to explicitly reflect on their own student agency skills. 


Self-Reflection on Summative Performance: Teachers provide 
opportunities for students to reflect after a project has been 
completed or on test performance or grades. 


Tying Self-Reflection to Evidence: Teachers require students to support 
their self-reflection with evidence, such as coursework or concrete examples. 


STUDENT-LED INSTRUCTION: Teachers provide students 
with opportunities to demonstrate agency by leading 
instruction on a particular skill or concept. 


Key Elements of Student-Led Instruction 

Student-Led Instruction on Content: Teachers provide students with 
opportunities to teach concepts or skills that are necessary to master 
as part of an academic course. 

Student-Led Instruction on Agency Skills: Teachers provide students 
with opportunities to teach agency skills that help foster student success 


on group or project work. 


The Menu of Teacher Practices on Student Agency was produced with support from Jobs for the Future’s Student-Centered Learning Research 


Collaborative and its funders. 


109b_10/18 


5: 


Maximizing Student Agency: Implementing and Measuring Student-Centered Learning Practices 


: Menu of Teacher Practices 
im) ATR int SUC PCHIANAMANI EIN CT 


AMERICAN INSTITUTES FOR RESEARCH* 


DEVELOPING RELATIONSHIPS: Teachers develop personal relationships with students to better understand their agency 
strengths, needs, and motivators. 
Key Elements of Developing Relationships 


Showing Commitment: Teachers follow through on commitments to students and show persistence in following up with students if they do not follow 
through on their commitments. 


Showing Empathy: Teachers make an effort to understand students’ circumstances when challenges arise and to help them think about how to overcome 
those circumstances or better prepare to face them next time. 


Showing Collaboration: Teachers harness their colleagues’ relationships with students when they have not yet established a relationship of their own 
or have trouble making the connection. 


FEEDBACK: Teachers provide students with feedback and scaffold the process of students asking for feedback. 
Key Elements of Feedback 


Direct Feedback: Teachers give students feedback to help improve particular projects and skills and to help students understand that feedback should 
be seen as an opportunity for growth rather than failure. 


Providing Opportunities to Ask for Feedback: Teachers scaffold the process of asking for feedback to ensure that students ask for feedback throughout 
the process rather than only when the teacher requires it. 


Providing Opportunities for Peer Feedback: Teachers scaffold the process of students working with peers to provide feedback as well as the process 
of students asking their peers for help before going to the teacher. 


GOAL SETTING: Teachers help students set goals to complete coursework while improving agency. 


Key Elements of Goal Setting 


Coursework Goals: Teachers work with students to set goals for project work so that students have benchmarks to guide themselves through a self-directed 
process of project completion. 


Agency Goals: Teachers work with students to identify challenges they face due to gaps in agency skills and help students develop plans for improving 
those challenges. 


INDIVIDUAL CONFERENCES: Teachers hold one-on-one meetings with students to discuss elements of student agency 
and its relationship to academic work. 
Key Elements of Individual Conferences 


Individual Conferences: Teachers may meet with students prior to beginning a project, during a project's implementation, or as a debrief after the fact to help 
students set goals for agency skills, problem-solve when agency is lacking, or reflect on agency strengths and challenges that they have demonstrated. 


STUDENT VOICE: Teachers provide students with opportunities to contribute to and provide feedback on key decisions 
in the classroom. 

Key Elements of Student Voice 

Selection of Resources: Teachers work with students to select resources that best meet their needs. 


Student Feedback: Teachers solicit student feedback and make changes to instruction or project parameters based on the feedback they receive. 


7/18 


109c_0 


The Menu of Teacher Practices on Student Agency was produced with support from Jobs for the Future’s Student-Centered Learning Research Collaborative 
and its funders. 


5: 


Maximizing Student Agency: Implementing and Measuring Student-Centered Learning Practices 


i ATR 


AMERICAN INSTITUTES FOR RESEARCH*® 


ASSESSMENT: Teachers design formative and summative 
assessments to evaluate student agency and/or to provide 
students with extrinsic motivation to build agency. 


Key Elements of Assessment 


Assessment Content: Assessments may be designed to explicitly measure 
student agency or may be broader assessments that require student agency 
to successfully complete. 


Assessment Data: Assessment data may include teacher observation, 
student self-reflection, and/or simple metrics, such as attendance 
or turning in homework. 


Sharing Assessment Results: Assessment results can be shared with 
students to help build agency dimensions, such as motivation. 


Student Design: Students can build agency by helping to design 
assessments of themselves and others. 


DIRECT INSTRUCTION: Teachers provide explicit instruction 
to develop skills related to student agency. 


Key Elements of Direct Instruction 


Mini Lesson: Teachers provide a whole-class lesson on a component 
of student agency. 


Small-Group Instruction: Teachers provide small-group instruction 
on components of student agency for which the group has a need. 


One-on-One Instruction: Teachers provide individual instruction 
on a component of student agency to address an immediate need. 


MODELING: Teachers model agency to demonstrate 
it to students in a meaningful context. 


Key Elements of Modeling 


Deliberate Modeling: Teachers integrate opportunities for modeling into 
their lesson plans as a means of demonstrating agency to students. 


Modeling as Part of Teachable Moments: Teachers model agency as situations 


arise that call upon them to exercise specific skills related to agency. 


POSITIVE REINFORCEMENT: Teachers provide positive 
reinforcement for demonstration of agency. 


Key Elements of Positive Reinforcement 


Verbal Praise: Teachers provide real-time verbal praise for demonstration 
of student agency. 
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% Teacher-Led Approaches 


Celebrations and Rewards: Teachers set up larger celebrations or rewards 
focused on providing positive reinforcement for student agency. 


Celebrating Failure: Teachers celebrate students who get things wrong 
to highlight that failure is an essential part of the learning process and 
overcoming failure is a key element of student agency. 


Letting Students Self-Reinforce: Teachers help students establish systems 
that they can use to provide positive reinforcement to themselves. 


SCAFFOLDING: Teachers provide students with tools, 
strategies, and resources to help scaffold students 
toward mastery of agency. 
Key Elements of Scaffolding 
Scaffolding Student Use of Time: Teachers provide parameters, tools, and 
strategies to help students accomplish a set of tasks in a predetermined 
length of time. 
Scaffolding Organization: Teachers provide tools such as agendas, logs, 
binders, and access to online platforms to help students keep track 
of assignments and make plans for completing all required work. 
Scaffolding Student Use of Resources: Teachers provide students with 
parameters and techniques to help them rely on a wide range of resources 
beyond asking the teacher for help. 
Scaffolding Group Interactions: Teachers provide students with strategies 
and tools to facilitate productive group work experiences. 


Scaffolding Problem Solving: Teachers provide students with strategies 
and resources to solve problems on their own when challenges arise. 


Scaffolding Overcoming Failure: Teachers provide students with strategies 
for and experience with overcoming failure and working toward success. 


VERBAL CUES: Teachers provide brief spoken prompts 
in real time to highlight or remind students of behaviors 
that demonstrate agency. 

Key Elements of Verbal Cues 

Reminders: Teachers provide students with brief, subtle reminders 


to demonstrate agency by reminding students of the desired behavior 
or praising students who are on track. 


Questions: Teachers ask students questions to lead them to necessary 
agency skills while also giving them ownership over the process. 


Explicit Links: Teachers make connections between the behaviors and 
skills they see and how they relate back to agency. 


The Menu of Teacher Practices on Student Agency was produced with support from Jobs for the Future’s Student-Centered Learning Research Collaborative and 


its funders. 
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Appendix B. Plan-Do-Study-Act Cycle Data 


School A 


Change idea: Administer a student self-reflection at the end of each week to measure mastery 
and resources used. 


Table B-1. School A Weekly Survey Respondents 


Freshman Sophomore Junior STAT Total 
Week Date Respondents Respondents Respondents Respondents Respondents 

1 2/23 49 100 38 56 243 
2 3/2 45 100 6 0 151 
3 3/9 25 103 0 48 176 
4 3/16 34 101 15 30 180 
5 3/23 2 49 0 5 56 
6 3/29 28 99 0 38 165 
if 4/13 52 177 34 53 316 
8 4/20 63 160 53 54 330 
9 4/27 44 132 55 14 245 
10 5/4 73 153 27 55 308 
11 5/11 61 138 62 38 299 
12 5/18 24 50 0 0 74 


Table B-2. School A Weekly Survey Responses (Weeks 1-6) 


Week 4 Week 5 Week 6 
March 16 March 23 March 29 


(n=180) (n=56) (n=165) 


In the past week, I... 


Asked a classmate 
for help ona 
concept that | 
didn't understand. 


Asked for a 
workshop on 

a concept that | 
didn't understand. 


179 73.7 76 50.3 119 676 124 689 32 57.1 112 67.9 


28 11.5 21 #139 21 #119 21 «117 6 10.7 19 11.5 
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Week 3 
ETCH) 


Week 4 
March 16 


Week 5 
March 23 


Week 6 
March 29 


(n=165) 


Attended a 

mandatory workshop 4, 469 25 166 33 188 32 178 4 «71 «26 158 
(a facilitator made 

me go). 

Attended a voluntary 

workshop (Iwent as 4g5 «28 «185 «38«(o216 SbT 150 i185 
without being 

forced). 

Ran a workshop 

for students who 

needed help on 12 4.9 11 7.3 13 7.4 5 2.8 1 1.8 9 5.5 
a concept that 

| understand. 

Asked a facilitator 

forindividual help = og 53.1 64 «42.4 «8246.6 «832A 2.9 8450.9 
on a concept that | 

didn’t understand. 

What supports did we use this past week that helped my learning? 

Conducting research. 78 32.1 32 21.22 28 159 36 20.0 0 0.0 26 15.8 
Doing an activity/ 

hands-on 101 41.6 44. 29.1 65 36.9 bf =~ StF 21 3/5 54 32.7 
assignment. 

petideniled 21 86 17 +113 #17 «9:7 «+14 #78 +8 00 9 55 
workshop. 

Watching a video. 81 33.3 47 31.1 53 30.1 76 42.2 2 3.6 49 29.7 
pole ties 89 366 51 338 63 358 52 289 22 393 58 35.2 
workshop. 

Reading an article. 74 ~=30.5 43 28.5 45 25.6 46 25.6 0 0.0 40 24.2 
A small group. 61 25.1 41 27.2 39 22.2 A1 22.8 9 16.1 41 24.8 
peelitator ted 61 251 41 272 39 #222 41 #228 #9 161 41 248 
workshop. 

A practice day. 2 0.8 18 11.9 33 18.8 27 15.0 32 57.1 32 19.4 
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Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 
Feb 23 March 2 ETCH) March 16 March 23 March 29 


(n=243) (n=151 (n=176) (n=180) (n=56) (n=165) 


n % 


How valuable did you find workshops in supporting your learning during this past week? 

21 8.6 17 «11.310 5.7 18 = 10.0 4 7.1 16 9.7 
28 11.5 14 9.3 15 8.5 14 7.8 9 16.1 16 9.7 
88 362 59 391 56 318 64 356 17 304 61 37.0 
74 305 37 245 66 375 57 31.7 13 £232 46 27.9 
32. 132 24 159 29 165 27 150 13 232 26 158 


oOo; RB] WO] nN] & 


How well did you do at advocating for your learning needs this week? 

1 9 3 10 6.6 3 1. 6 3.3 4 7.1 8 48 
2 16 6.6 8 5.3 14 8.0 id 6.1 3 5.4 13 7.9 
3 107 440 55 364 62 352 81 450 17 304 63 38.2 
4 79 325 54 358 61 347 57 31.7 17 304 53 32.1 
5 31 132 24 159 36 205 25 139 15 268 28 = 17.0 


When it comes to the concepts learned in the past week, | feel like | am... 


proficient. 66 272 59 39.1 68 386 54 300 20 35.7 79 48.2 


still working on 


. ae 59.7 78 51.7 88 50.0 109 606 27 482 76 46.3 
becoming proficient. 


struggling to 


. 13.2 14 9.3 20 11.4 17 9.4 9 16.1 9 5.5 
become proficient. 


Table B-3. School A Weekly Survey Responses (Weeks 7-12) 


Week 7 Week 8 Week 9 Week 10 Week 11 Week 12 
April 13 April 20 April 27 May 4 May 11 May 18 


(n=316) (n=74) 


In the past week, l... 


Asked a classmate 
for help ona 
concept that | 
didn’t understand. 


Asked for a 
workshop on a 
concept that | 
didn’t understand. 


213 67.4 226 685 158 645 207 67.2 199 666 56 75.7 


39 123 37 0611.2 ©6330 13.5 «6331 10.1 «6400S 13.4 3 4.1 
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Week 7 Week 8 Week 9 Week 10 Week 11 Week 12 
April 13 April 20 April 27 May 4 May 11 May 18 


(n=74) 


Attended a 
mandatory workshop 
(a facilitator made 
me go). 


53 168 66 200 58 23.7 77 250 88 294 16 21.6 


Attended a voluntary 
workshop (I went 
without being 
forced). 


Ran a workshop 

for students who 

needed help on 6 1.9 19 5.8 10 4.1 10 3.2 10 3.3 2 2.7 
a concept that 

| understand. 


Asked a facilitator 
for individual help 
on a concept that | 
didn’t understand. 


152 481 165 500 118 482 160 519 147 492 34 45.9 


What supports did we use this past week that helped my learning? 


Conducting 


73, (23.1 75 22.7 59 24.1 56 18.2 85 28.4 15 20.3 
research. 


Doing an activity/ 
hands-on 126 39.9 107 324 100 408 115 373 112 375 31 441.9 
assignment. 


A student-led 
workshop. 


Watching a video. 121 383 131 39.7 55 224 107 347 92 308 15 20.3 


13 4 370 11.211 4.5 11 3.6 11 3.7 3 Al 


pwhole-tlass 109 345 127 385 93 380 111 360 125 418 21 284 
workshop. 

Reading an article. 96 304 120 364 71 29.0 91 29.5 88 29.4 9 12.2 
A small group. 45 142 50 152 50 204 51 166 61 204 11 149 


A facilitator-led 
workshop. 


A practice day. 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 


45 142 50 152 50 204 51 166 61 204 11 «149 


Maximizing Student Agency: Implementing and Measuring Student-Centered Learning Practices 


Week 7 Week 8 Week 9 Week 10 Week 11 Week 12 
April 13 April 20 April 27 May 4 May 11 May 18 


(n=330) (n=245) (n=308) (a:)) (n=74) 


How valuable did you find workshops in supporting your learning during this past week? 


1 29 9.2 34 103 24 98 30 9.7 20 6.7 18 = 24.3 
2 65 206 66 200 42 171 #62 201 48 161 #15 20.3 
3 136 43.0 143 433 95 388 135 438 135 45.2 25 33.8 
4 86 272 87 264 84 343 81 263 96 321 16 21.6 
5a 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 


How well did you do at advocating for your learning needs this week? 


1 % 5.1 18 5:5 12 4.9 17 5.5 103.3 8 10.8 
2 57 618.0 65206158 «63370151 «520 169 «86390 «613.0 «15 = 20.3 
3 138 43.7 159 482 109 445 138 448 143 47.8 30 40.5 
A 105 33.2 101 306 87 355 101 32.8 107 358 21 284 
5a 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 


When it comes to the concepts learned in the past week, | feel like | am... 


proficient. 134 42.5 132 404 117 478 117 380 128 428 26 35.1 


still working on 


. 7 161 51.1 163 498 108 441 164 532 145 485 38 451.4 
becoming proficient. 


struggling to 


sd 6.3 32 9.8 20 8.2 Dif 8.8 26 8.7 10 3913.5 
become proficient. 


*The questions “How valuable did you find workshops in supporting your learning during this past week” and “How well did you do at 
advocating for your learning needs this week?” offered 1-5 Likert scales in weeks 1-6 and 1-4 Likert scales in weeks 7-12. In both cases, 
1 corresponds to the worst possible rating and 5 or 4 corresponds to the best possible rating. 
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School B 


Change idea: Provide students with additional resources and feedback so they can revise and 
improve their work. 


Table B-4. School B IAKT Survey Responses 


February (n=93) March (n=202) May (n=178) 


Number Percent Number Percent Number Percent 


To what extent did responding to teacher feedback left in Echo contribute to your learning while completing this IAKT? 


Did not use 4 43 27 13.4 24 13.5 
Did not contribute 4 43 6 3.0 6 3.4 
Contributed some 30 32.3 68 33.7 60 33.7 
Contributed a lot 55 59.1 101 50.0 88 49.4 


To what extent did responding to peer feedback contribute to your learning while completing this IAKT? 


Did not use 32 34.4 53 26.2 AT 26.4 
Did not contribute 11 11.8 19 9.4 18 10.1 
Contributed some 34 36.6 Al 20.2 35 19.7 
Contributed a lot 16 17.2 89 44.1 78 43.8 


To what extent did using the resources available in the project briefcase contribute to your learning while 
completing this IAKT? 


Did not use 14 14.9 20 9.9 18 10.1 
Did not contribute 4 43 6 3.0 6 3.4 
Contributed some 35 37.2 99 49.0 88 49.4 
Contributed a lot Al 43.6 77 38.1 66 37.1 


To what extent did conducting independent research online contribute to your learning while completing this IAKT? 


Did not use 7 Led 13 6.4 12 6.7 
Did not contribute 3 3.2 0 0.0 0 0.0 
Contributed some 16 17.0 20 9.9 18 10.1 
Contributed a lot 68 72.3 169 83.7 148 83.2 


To what extent did teacher-led workshops contribute to your learning while completing this IAKT? 


Did not use 37 41.6 106 52.5 95 53.4 
Did not contribute 15 16.9 47 23.3 Al 23.0 
Contributed some 27 30.3 35 17.3 30 16.9 
Contributed a lot 10 142 14 6.9 12 6.7 
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February (n=93) March (n=202) May (n=178) 


Number Percent Number Percent Number Percent 


To what extent did teacher-led conferences contribute to your learning while completing this IAKT? 


Did not use 42 46.7 107 53.0 96 53.93 
Did not contribute 14 15.6 Al 20.3 36 20.22 
Contributed some 25 27.8 40 19.8 34 19.1 
Contributed a lot 9 10.0 14 6.9 12 6.74 


Table B-5. School B Growth Mindset Survey Responses 


February (n=125) March (n=101) 


Number Percent Number Percent 


How true is the following about you? My intelligence is something that | can’t change very much. 


Not at all true 59 47.2 50 49.5 
A little true 28 22.4 23 22.8 
Somewhat true 29 23.2 22 21.8 
Mostly true 7 5.6 6 5.9 
Completely true 2 1.6 0 0.0 


How true is the following about you? Challenging myself won’t make me any smarter. 


Not at all true 72 57.6 67 66.3 
A little true 32 25.6 17 16.8 
Somewhat true 11 8.8 14 13.9 
Mostly true 9 7.2 3 3.0 
Completely true 1 0.8 0 0.0 


How true is the following about you? There are some things | am not capable of learning. 


Not at all true 38 30.4 44 43.6 
A little true 39 31.2 24 23.8 
Somewhat true 25 20.0 21 20.8 
Mostly true 13 10.4 7 6.9 
Completely true 10 8.0 5 5.0 


How true is the following about you? If | am not naturally smart in a subject, | will never do well in it. 


Not at all true 56 44.8 56 55.5 
A little true 40 32.0 23 22.8 
Somewhat true 17 13.6 17 16.8 
Mostly true 7 5.6 4 4.0 
Completely true 5 4.0 1 1.0 
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School C—Teacher 1 


Change idea: Waive zero-grade policy for students who seek out extra help. 


Table B-6. School C Writing Agency Survey Responses 


February (n=22) March (n=22) May (n=22) 


Number Percent Number Percent Percent 


| would describe myself as a writer. 


Agree or Strongly Agree 12 54.6 12 54.6 15 68.2 

Disagree or Strongly Disagree 10 45.5 10 45.5 7 31.8 
| believe that, with practice, my writing will improve. 

Agree or Strongly Agree 14 63.6 16 72.7 16 72.7 

Disagree or Strongly Disagree 8 36.4 6 27.3 6 27.3 


When | am struggling with a writing assignment, | think about attending a support session before or after school. 


Agree or Strongly Agree 3 13.6 10 45.5 13 59.1 

Disagree or Strongly Disagree 19 86.4 12 54.5 9 40.9 
My grade is a reflection of my effort in English I. 

Agree or Strongly Agree 18 81.8 19 86.4 21 95.5 

Disagree or Strongly Disagree 4 18.2 3 13.6 1 4.5 


Table B-7. School C Attendance at Workshops 


Date of Workshop Number Percent 
March 12 0 0.0 
March 14 5 22.7 
March 19 4 18.2 
March 21 14 63.6 
March 23 0 0.0 

April 9 2 9.1 
April 11 2 9.1 
April 13 3 13.6 
April 16 1 45 
May 8 0 0.0 
May 10 0 0.0 
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School C—Teacher 2 


Change idea: Provide students with personalized verbal feedback. 


Table B-8. School C Feedback Survey Responses 


Week 1 Week 2 Week 3 Week 4 
February 20 February 26 ETCH) March 21 


et) (n=63) (n=53) (n=56) 


Number Percent Number Percent Number Percent Number Percent 

Receiving feedback helps me improve my work. 
Yes 49 74.24 56 88.89 49 92.45 50 89.29 
No 17 25.76 7 11.11 4 7:55 6 10.71 


| know how to use feedback. 
Yes 55 83.33 53 84.13 50 94.34 50 89.29 
No 11 16.67 10 15.87 3 5.66 6 10.71 


When | look at feedback from teachers, | feel... 


Hopeful 33 50.0 30 47.6 27 50.9 29 51.8 
Confident 29 43.9 28 44.4 28 52.8 30 53.6 
Capable 33 50.0 34 54.0 30 56.6 28 50.0 
Positive 34 515 35 55.6 28 52.8 27 48.2 
Challenged 35 53.0 30 47.6 27 50.9 34 60.7 
Anxious 24 36.4 22 34.9 19 35.9 21 31.9 
Frustrated 23 34.9 24 38.1 14 26.4 16 28.6 
Like a Failure 16 24.2 11 LES 9 17.0 alg 19.6 
Over-whelmed 21 100.0 16 25.4 14 26.4 13 23.2 
Confused 18 2453 18 28.6 10 18.9 15 26.8 
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School D—Teacher 1 


Change idea: Provide students more choice in support resources. 


Table B-9. School D Resources: Survey A Responses 


Week 1 Week 2 
February 23 March 3 
(n=22) (n=38) 


Number Percent Number Percent 


| would grade my effort an... 


A 9 40.91 13 34.21 
B 6 27.27 11 28.95 
C 6 27.27 9 23.68 
D 1 4.55 2 5.26 
Did you complete all assignments for this week? 
Yes 12 54.55 28 73.68 
No 10 45.45 10 26.32 
Did this week's test prep activities provide you with relevant test prep/practice? 
Yes 16 72.73 31 81.58 
No 5 22.73 1 2.63 
| used the following resources this week: 
Workshop (forced) 8 36.36 15 39.47 
Study guide 6 27.27 20 52.63 
Online practice tests 6 27.27 20 52.63 
Review tools 6 27.27 20 52.63 
Online Web sources and worked independently 12 54.55 16 42.11 
Outside tutoring (after school) 1 4.55 4 10.53 
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Table B-10. School D Resources: Survey B Responses 


Week 3 Week 4 
April 20 April 27 
(ie?) (n=40) 


This resource This resource This resource 
| used this was the | used this was the | used this was the 
resource. most helpful. resource. most helpful. resource. most helpful. 


n n 
Tutoring 6 21.4 2 7.1 4 10.0 1 2.5 1 6: 0 0.0 
Edcite 23. 82.1 14 500 31 77.5 25 = 62.5 8 53.3 4 26.7 
Quizlet 11. 39.3 2 tA 10 = 25.0 3 15 1 6.7 0 0.0 
Study Guide 8 286 1 36 10 250 2 5.0 5 333 0 0.0 
Study Island 3 107 O 0.0 4 100 0 0.0 2 133 0 0.0 
Crash Course 3 10.7 1 3.6 4 100 0 0.0 1 6.7 0 0.0 
Individual work 14 500 3 10.7 20 500 4 10.0 8 533 3 20.0 
Classroom 

presentations 11 33930 «4 143 15 375 2 5.0 7 467 2 13.3 
or slides 


School D—Teacher 2 


Change idea: Provide students more choice in support resources. 
Table B-11. School D Resources: Survey C Responses 


Week 1 Week 2 Week 3 
April 6 April 13 April 20 
(ia) (iie?4s)) (it) 


Number Percent Number Percent Number Percent Number Percent 


| used slideshows/notes 
this week to help me learn 20 95.2 25 89.3 6 66.7 20 86.96 
the content. 


| used videos provided 
by the teacher to help 1 4.8 2 fell 0 0.0 2 8.7 
me learn the content. 


| used a whole- class 
workshop with the teacher 16 76.2 19 67.9 2 22.2 15 65.22 
to help me learn the content. 


| used a Classkick 
workshop on my own to 13 61.9 13 46.4 1 11.1 14 60.87 
help me learn the content. 
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Week 1 Week 2 Week 3 
April 6 April 13 April 20 
(n=21) ((iie?43)) (it) 


Number Percent Number Percent Number Percent Number Percent 


| used individual help from 
the teacher to help me 11 52.4 12 42.9 0 0.0 8 34.78 
learn the content. 


| used a resource | found 
on my own to help me 3 14.3 1 3.6 5 55.6 3 13.04 
learn the content. 


School D—Teacher 3 


Change idea: Provide students more choice in support resources. 


Table B-12. School D Agency Survey Responses 


Week 1 Week 2 Week 3 Week 4 
January 12 January 19 January 26 February 2 
(ae )) ‘(ie h)) (ae) ‘(fa?4s)) 


Number Percent Number Percent Number Percent Number Percent 


| would grade my effort an... 


A 15 45.5 10 28.6 14 42.4 5 20.0 
B 10 30.3 11 31.4 8 24.2 7 28.0 
C 8 24.2 9 25.7 if 21.2 10 40.0 
D 0 0.0 2 5.7 2 6.1 2 8.0 


Did you complete all assignments for this week? 


Yes 17 51.52 23 65.7 23 69.7 15 60.0 


No 16 48.48 12 34.3 10 30.3 10 40.0 


Did this week's test prep activities provide you with relevant test prep/ practice? 


Yes - - 30 85.7 29 87.9 21 84.0 


No - - 4 11.4 1 9.1 3 12.0 
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School D—Teacher 4 


Change idea: Provide one-on-one conferencing on IAKT assignments. 


Table B-13. School D Student Writing Grades 


Score of 1 Score of 2 Score of 3 
Number Percent Number Percent Number Percent 
Development 60 79.7 15 20.3 0 0.0 
pet PSSM reanization 35 47.3 36 48.7 3 4.1 
Language 52 68.9 23 31.1 0 0.0 
Assignment 1 Draft Development 56 74.3 14 18.9 5 6.8 
(n=75) Organization 55 73.0 17 23.0 3 41 
Apa tO Language 53 70.3 19 25.7 3 41 
Development 43 56.8 26 35.1 6 8.1 
— ene Giaaizaton 43 56.8 17 23.0 15 20.3 
Language 43 56.8 21 28.4 11 14.9 
Assignment 2 Draft Development 60 79.7 13 17.6 2 2.7 
(n=75) Organization 52 68.9 20 27.0 3 41 
peulge Language 55 73.0 19 25.7 1 14 
Development 60 79.7 12 16.2 3 4.1 
i ae - Se aanon 52 68.9 14 18.9 9 12.2 
Language 52 68.9 19 25.7 4 5.4 
Assignment 3 Draft Development 50 66.2 24 32.4 1 1.4 
(n=75) Organization 45 59.5 26 35.1 4 5.4 
May tt Language 44 58.1 28 37.8 3 4.1 
Development 20 27.0 25 33.8 29 39.2 
peal Final Organization 25 33.8 23 Sit 26 35.1 
Language 25 33.8 24 32.4 25 33.8 
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